Search for: All records

Reinforcement learning (RL) methods can be used to develop a controller for the heating, ventilation, and air conditioning (HVAC) systems that both saves energy and ensures high occupants’ thermal comfort levels. However, the existing works typically require on-policy data to train an RL agent, and the occupants’ personalized thermal preferences are not considered, which is limited in the real-world scenarios. This paper designs a high-performance model-based offline RL algorithm for personalized HVAC systems. The proposed algorithm can quickly adapt to different occupants’ thermal preferences with a few thermal feedbacks, guaranteeing the high occupants’ personalized thermal comfort levels efficiently. First, we use a meta-supervised learning algorithm to train an occupant's thermal preference model. Then, we train an ensemble neural network to predict the thermal states of the considered zone. In addition, the obtained ensemble networks can indicate the regions in the state and action spaces covered by the offline dataset. With the personalized thermal preference model updated via meta-testing, model-based RL is used to derive the optimal HVAC controller. Since the proposed algorithm only requires offline datasets and a few online thermal feedbacks for training, it contributes to a more practical deployment of the RL algorithm to HVAC systems. We use the ASHRAE database II to verify the effectiveness and advantage of the meta-learning algorithm for modeling different occupants’ thermal preferences. Numerical simulations on the EnergyPlus environment demonstrate that the proposed algorithm can guarantee personalized thermal preferences with a slight increase of power consumption of 1.91% compared with the model-based RL algorithm with on-policy data aggregation. 

The model of personalized thermal comfort can be learned via various machine learning algorithms and used to improve the individuals’ thermal comfort levels with potentially less energy consumption of HVAC systems. However, the learning of such a model typically requires a substantial number of thermal votes from the considered occupant, and the environmental conditions needed for collecting some votes may be undesired by the occupant in order to obtain a model with good generalization ability. In this paper, we propose to use a meta-learning algorithm to reduce the required number of personalized thermal votes so that a personalized thermal comfort model can be obtained with only a small number of feedback. With the learned meta-model, we derive a method based on the backpropagation of neural networks to quickly identify the best environmental and personal conditions for a specific occupant. The proposed identification algorithm has an additional advantage that the thermal comfort, indicated by the mean thermal sensation value, improves incrementally during the data collection process. We use the ASHRAE global thermal comfort database II to verify that the meta-learning algorithm can achieve an improved prediction accuracy after using 5 thermal sensation votes from an occupant to make adaptations. In addition, we show the effectiveness of the fast identification algorithm for the best personalized thermal environmental conditions with a thermal sensation generation model built from the PMV model.